Inference rule determining method and inference device

ABSTRACT

An inference rule determining process according to the present invention sequentially determines, using a learning function of a neural network model, a membership function representing a degree which the conditions of the IF part of each inference rule is satisfied when input data is received to thereby obtain an optimal inference result without using experience rules. The inventive inference device uses an inference rule of the type &#34;IF . . . THEN . . .&#34; and includes a membership value determiner (1) which includes all of IF part and has a neural network; individual inference quantity determiners (21)-(2r) which correspond to the respective THEN parts of the inference rules and determine the corresponding inference quantities for the inference rules; and a final inference quantity determiner which determines these inference quantities synthetically to obtain the final results of the inference. If the individual inference quantity determiners (2) each has a neural network structure, the non-linearity of the neural network models is used to obtain the result of the inference with high inference accuracy even if an object to be inferred is non-linear.

This is a continuation of application Ser. No. 07/459,815, filed Jan.17, 1990, now U.S. Pat. No. 5,168,549.

TECHNICAL FIELD

The present invention relates to inference rule determining methods andinference devices in controling apparatus, in the inference of an expertsystem, and in pattern recognition based on input data.

BACKGROUND ART

In order to describe a conventional technique, first, the basic fuzzyinference will be outlined by taking as an example fuzzy control used inapparatus control, etc.

In a control system which relates to the evaluation of human beings, theoperator can determine a final manipulated control variable using avariable which the operator has determined subjectively and/orsensuously, for example, "large", "middle", "tremendously", or "alittle" (which is hereinafter referred to as a fuzzy variable). In thiscase, the operator determines the manipulated variable from the inputvariables on the basis of his control experience. An inference deviceusing fuzzy control assigns an input fuzzy number to an IF part of aninference rule in accordance with the inference rule of "IF . . . THEN .. ." type and determines an output fuzzy number of the THEN part from afitting grade (membership value) indicative of the extent to which theinference rule is satisfied. The actual manipulated variable can beobtained by taking the center of gravity value, etc., of the outputfuzzy number.

One of the conventional control methods using fuzzy inference is fuzzymodeling disclosed, for example, in Gean-Taek Kang, Michio Sugano;"fuzzy modeling" SOCIETY OF INSTRUMENT AND CONTROL ENGINEERS PAPERS,Vol. 23, No. 6, pp. 650-652, 1987. In the control rule of the fuzzymodeling, the IF part is constituted by a fuzzy proposition and the THENpart is constituted by a regular linear equation between inputs andoutputs. If a timing lag of first order tank model is considered, forexample, the control rule among a control error e, its change in errorde and a control output (manipulated variable) u is given by

If e is Zero and de is Positive Medium

Then u=0.25 e+1.5 de

A plurality of such inference rules are prepared, all of which arereferred to as control rules. Zero, Positive Medium, etc., are each alabel or a fuzzy variable (fuzzy number) used to described the rules.FIG. 1 illustrates one example of a fuzzy variable. In FIG. 1, NBdenotes Negative Big; NM, a negative Medium; NS, a Negative Small; ZO, aZero; PS, Positive Small; PM, a Positive Medium; and PB, Positive Big. Afunction indicative of a fuzzy number F on X is referred to as amembership function μ_(f) ( ) and the function value of x⁰ is referredto as a membership value μ_(F) (X⁰). The general form of the controlrules is given by

    R.sup.s : If x.sub.1 is A.sub.1.sup.s, x.sub.2 is A.sub.2.sup.s, . . . , x.sub.m is A.sub.m.sup.s

    Then y.sup.s =c.sub.0.sup.s +c.sub.1.sup.s x.sub.1 +c.sub.2.sup.s x.sub.2 +. . . +c.sub.m.sup.s x.sub.m

where R⁵ indicates a s^(th) rule; x_(j), an input variable; A_(j) ^(s),a fuzzy variable; y^(s), an output from the s^(th) rule; and c^(s), aTHEN part parameter. The result of inference for an input (x₁ ⁰, x₂ ⁰, .. . , x_(m) ⁰) is given by ##EQU1## where n is the number of rules, andw^(s) is a fitting grade at which the input (x₁ ⁰, x₂ ⁰, . . . , x_(m)⁰) is adapted to the IF part of the S^(th) rule. W^(s) is given by##EQU2## where the membership value in the x⁰ in the fuzzy variableA_(j) ^(s) is μ_(Aj) ^(s) (x_(j)). The identification of a fuzzy modelincludes a two-stage structure, namely, identification of the structureof the IF and THEN parts and identification of the IF and THEN parts.The conventional identifying process includes the steps of (1) changingthe fuzzy proposition of the IF part to a proper proposition, (2)changing W^(s) in a constant manner, (3) searching only the actuallyrequired ones of the input variables of the THEN part using a backwardelimination method, (4) calculating parameters of the THEN part usingthe method of least squares, (5) repeating the steps (2)-(4) todetermine an optimal parameter, (6) changing the fuzzy proposition ofthe IF part and (7) returning to the step (2) where an optimal parameteris repeatedly searched under the conditions of a new fuzzy proposition.Namely, this method can be said to be a heuristic method-likeidentifying algorithm.

A conventional inference device includes a fuzzy inference device, forexample, shown in FIG. 2 in which reference numeral 1a denotes a datainput unit (including a measured value and a value evaluated by humanbeing); 2a, a display command unit; 3a, a fuzzy inference operationunit; 4a, an inference result output unit; and 5a, a display. Thedisplay 2a is composed of a keyboard, and the fuzzy inference operationunit 3a is composed of a digital computer. The input data to the digitalinput unit 1a is subjected to inference operation at the fuzzy inferenceoperation unit 3a. Thus, the operation unit 3a outputs the result of theinference and simultaneously displays a list of inference rules, a listof fuzzy variables and the states of use of various inference rules onthe display 5a. In a conventional inference device such as that shown inFIG. 2, the inference rules of the fuzzy inference rules and the fuzzyvariables as input data are fixed as constant values in the fuzzyinference operation unit 3a and have no function of changing the fuzzyvariables.

Since an algorithm of determining a membership function is based onheuristic method in the conventional inference rule determining methodas well as in the fuzzy modeling, it is complicated and the number ofparameters to be determined is very large. Therefore, optimal inferencerules cannot be obtained easily at high speed.

Since an influence device such as that shows in FIG. 2 has no functionof learning inference rules, the characteristic of the inferencevariable changes with time, so that it cannot cope with the situationthat the inference accuracy would be deteriorated according to theinference rules set initially. Assume, for example, that there is theinference rule "If it is hot, then control diarie a so as to be in levelB." Since the obscure concept "hot" varies from person to person as wellas season to season, satisfactory control cannot be achieved unless theinference device has a learning function and can change the inferencerules adaptively in accordance with situations under which the device isused. Furthermore, there are actually many cases where non-linearinference is required, so that a method of performing linearapproximation described with reference to the above conventional examplehas a limit to improvement in the inference accuracy.

SUMMARY OF THE INVENTION

In order to solve the above problems, it is a first object of thepresent invention to provide an inference rule determining process forautomatically determining a fuzzy number contained in the inferencerules described in the form "IF . . . THEN. . ." using a learningfunction without relying upon experience.

It is a second object of the present invention to provide an inferencedevice which is capable of coping with an inference problem at highspeed even if the problem is non-linear, using the non-linearity of aneural network model. In the inference device, a membership valuedeterminer constituting the neural network infers from the inputvariables a membership value corresponding to the IF part of eachinference rule. The individual inference quantity determiner of theinference device infers from the input variables an inference quantitycorresponding to the THEN part of the inference rule. The finalinference quantity determiner makes a synthetic determination based onthe inference quantity and the membership value inferred for each of therules for the input variables and obtains a final inference quantity.The present invention is the inference device which functions asdescribed above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates fuzzy variables; FIG. 2 is a schematic of aconventional inference device; FIGS. 3 and 4 are schematics of neuralnetwork models which constitute the membership value determiner and theindividual inference quantity determiner; FIG. 5 is a schematic of alinear signal processor; FIG. 6 is a schematic of a non-linear signalprocessor; FIG. 7 is an input-output characteristic diagram of athreshold processor; FIG. 8 illustrates a process for automaticallydetermining a membership function using a neural network model; FIG. 9is a schematic of an inference device of one embodiment in the presentinvention; FIG. 10 is a schematic of input/output data used fordescribing the operation of the FIG. of embodiment 1; FIG. 11illustrates the membership value of each learning data in the IF part ofthe FIG. 9 embodiment; FIG. 12 illustrates an inference error producedwhen the number of input variables is reduced; FIG. 13 illustrates theresult of inferring the inference quantity of the FIG. 9 embodiment;FIG. 14 illustrates the result of the inference according to the presentinvention applied to the problem of inferring the COD density in OsakaBay.

BEST MODE FOR CARRYING OUT THE INVENTION

The features of the inventive method lies in sequentially determiningmembership functions using a learning function. Various methods ofrealizing the learning function are conceivable. In a first embodiment,a learning algorithm of a neural network model is used. Beforedescription of the embodiment, a neural network model and a neuralnetwork model learning algorism will be described.

The neural network model is a mathematical network which has gotten ahint from the combination of cranial neural cells. Inference rules canbe determined without relying on experience by determining the intensityof connection of units constituting the network using sequentiallearning.

The network of FIG. 3 is a kind of a neural network model. In FIG. 3,reference numeral 100 denotes a multi-input-single output signalprocessor; and 101, input terminals of the neural network model. FIG. 4shows an example of a 3-stage structured neural network model whichreceives four input variables and outputs a single output value. Thereis no interconnection in each stage and a signal is propagated to anupper stage only. FIG. 5 specifically illustrates the structure of alinear signal processor which basically includes a linear computationsection of the multi-input-single output signal processors 100constituting the neural network model. In FIG. 5, reference numeral 1001denotes each of input terminals of the multi-input-single output signalprocessor 100; 1002, a memory which stores weighting coefficients givento the corresponding inputs from the input terminals 1001; 1003,multipliers which multiply the inputs from the input terminals 1001 bythe corresponding weight coefficients in the memory 1002; and 1004, anadder which adds the respective outputs from the multipliers 1003.Namely, the multi-input-single output signal processor 100 of FIG. 5calculates

    y=Σc.sub.i x.sub.i

where x_(i) is an input value to the corresponding one of the inputterminals 1001 and c_(i) is the corresponding one of the weightcoefficients stored in the memory 1002. FIG. 6 specifically illustratesthe structure of a non-linear signal processor, which performs anon-linear operation, of the multi-input-single output signal processors100 which constitute the neural network model. In FIG. 6, referencenumeral 1000 denotes the linear signal processor described withreference to FIG. 5; and 2000, a threshold processor which limits theoutput from the linear signal processor to a value in a predeterminedrange. FIG. 7 illustrates the input-output characteristic of thethreshold processor 2000. For example, the input/output characteristicof the threshold processor 2000 which limits the output from the linearsignal processor 1000 to within the range (0, 1) is representedmathematically by

    OUT=1/(1+exp(-IN))

where IN and OUT are the input and output to and from the thresholdprocessor 2000.

The input/output relationship in the neural network is represented by

    Y=NN(x)                                                    (1)

In the description of the embodiment, the model size is represented bythe k-layer |μ₀ x μ₁ x . . . x μ_(k) | where μ_(i) is the number ofneuron models of each of the input, middle and output layers. The aboverefers to a general description of the neural network model.

The learning algorithm of the neural network model will now bedescribed. Various learning, algorithms are proposed, one of which is abackpropagation (D. E. Rumelhart, G. E. Hinton and R. J. Williams,"Learning Representations by Back-Propagating Errors," Nature, Vol. 323,pp. 533-536, Oct. 9, 1986). The mathematical verification andformulation of this algorithm are left to references, but it isbasically described as follows. First, a multiplicity of sets of inputvariables as learning data and percentage values belonging to therespective inference rules inferred by other means are prepared. Inputdata is input to the neural network model. Initially, an appropriatevalue, for example a random number value, is input in the memory 1002.The error between the output value from the neural network model and thepercentage value belonging to the inference rules is calculated. Sincethe error is a function of the weight coefficients in the memory 1002,the weight coefficient is corrected in the direction of weightdifferentiation with respect to the error. Namely, the error isminimized and learning is performed by correcting the weightssequentially in accordance with the Steepest Decent Method. This is thesummary of the learning method using Backpropagation Algorithm.

An inference rule determining process and the inference quantitycalculating process according to the present invention will be describedusing the neural network model and learning algorithms.

Step 1

Only input variables having high correlation to an inference value areselected and unnecessary input variables are removed. If thedetermination is performed, for example, using the neural network model,first, respective input variables x_(j) where j=1, 2, . . . k areassigned to the neural network model and learning is performed such thatan inference value y_(i) is obtained. Similar learning is performed bydecrementing the number of input variables by one. Similarly, thebackward elimination method is repeated. The difference between eachoutput of the neural network model which has completed the correspondinglearning and the inference value y_(i) is compared using the error sumof squares as an evaluation index, and only the input variables (X_(j)where j=1, 2, . . . m, where m≦k) to the neural network model presentwhen the error is minimized are selected.

Step 2

The input/output data (x_(i), y_(i)) is divided into n_(t) data segmentsfor identifying the structure of an inference rule (hereinafterdescribed as TRD) and n_(o) inference rule evaluation data segments(hereinafter described as CHD) where n=n_(t) +N_(o).

Step 3

Each TRD is divided into optimal r sub-segments using regular clusteringmethod. Each set of learning data segments to which the TRD is dividedis R^(s) where s=1, 2, . . . , r, and the learning data on R^(s) is(x_(i) ^(s), Y_(i) ^(s)) where i=1, 2, . . . , (n_(t))^(S) where(n_(t))^(s) denotes the number of data of the TRD in each R^(S). Ther-division of an m-dimensional space means that the number of inferencerules is r.

Step 4

The IF part structure is identified. X_(i) is an input value to theinput layer of the neural network model and the output value from theoutput layer is given by ##EQU3## A neural network model is learned suchthat the grade w_(i) ^(s) of each learning data (x_(i)) belonging toeach R^(s) is inferred. The output value from the neural network modelafter learning is the membership value of the IF part. Namely, themembership function is given by

    μ.sub.A.sup.s (x.sub.i)=W.sub.i.sup.s

where i=1, 2, . . . , n

Step 5

The structure of the THEN part is identified. The THEN part structuremodel of each inference rule is A represented by the input-outputrelation (1) of the neural network model and the input values x_(i)^(s), . . . , x_(im) ^(s) where i=1, 2, . . . , (n_(t) ^(s)) and theoutput value y_(i) ^(s) are allocated. A neural network model whichinfers the inference quantity by learning is identified. Substitutingthe CHD input values x_(il) ^(s), . . . , x_(im) ^(s) where i=1, 2, . .. , n_(c) into the resulting neural network model and the error sum ofsquares O^(s) is calculated by ##EQU4## There is another idea tocalculate θ_(m) ^(s) with the weight; this is ##EQU5##

Step 6

Only the input variables contributing to the inference quantityinferring equation of the THEN part of each inference rule are selectedusing variable reducing method. Only any one of m input variables isremoved and a neural network model is then identified for each THEN partusing the TRD similarly to step 5. The inferred error sum of squaresθ_(m-1) ^(sp) of the inference quantity obtained when the CHD is used iscalculated as follows: ##EQU6## From the above expression, it will beseen to what extent the removed input variable is important to determinethe THEN part. If

    θ.sub.m.sup.s >θ.sub.m-1.sup.sp,

the input variable x^(p) is discarded because it is considered to be notimportant.

Step 7

An operation similar to step 5 is performed by using the m remaininginput variables. Subsequently, steps 5 and 6 are repeated and thecalculation is stopped when no equation (2) holds for all the inputvariables. The optimal neural network model is the one where O^(s)minimum.

The steps 1-7 determine the IF and THEN parts for each inference ruleand the identification of the structure of the fuzzy inference ruleends.

Step 8

The quantity of inference y_(i) is obtained by ##EQU7## where y^(s)shows a presumed value obtained by substituting CHD into the optimalneural network model obtained at step 7.

If the neural network model includes a collection of non-linearcharacteristics of FIG. 7, the inference of a non-linear object isautomatically possible by this learning process, of course.

Thus, if the inference rule is determined by the inventive process, theinference rules are obtained easily and accurately without using theheuristic method since the inference rules are determined sequentiallyusing the learning function even if the object to be inferred is in anon-linear relationship or in an obscure input/output relationship.

The inventive process described above will now be described usingspecific experimental values. In the particular experiment, theinference rule and the quantity of inference were determined usingsimple numeral examples shown in Tadashi Kondo, Revised GMDH AlgorithmEstimating Degree of the Complete Polynomial, SOCIETY OF INSTRUMENT ANDCONTROL ENGINEERS PAPERS, Vol. 22, No. 9, pp. 928-934, 1986. The processfor determining the inference rule will be described below.

Steps 1, 2

FIG. 10 shows input/output data where data numbers 1-20 denote data fordetermining the structure of an inference rule (TRD) used in learningand data numbers 21-40 denote evaluation data (CHD). Therefore, n_(t)=n_(c) =20 and m=2. Input variables were selected by learning 15,000times in a three-layered (3×3×3×1) neural network model. The results areshown in Table 1.

                  TABLE 1                                                         ______________________________________                                                       ERROR SUM OF SQUARES                                           ______________________________________                                        when all variables were used                                                                   0.0007                                                       when x.sup.1 was removed                                                                       0.3936                                                       when x.sup.2 was removed                                                                       0.1482                                                       when x.sup.3 was removed                                                                       0.0872                                                       when x.sup.4 was removed                                                                       0.0019                                                       ______________________________________                                    

The error sum of squares is not so much influenced even if the inputvariable x⁴ is added or otherwise removed. Therefore, it is determinedthat it does not virtually influence the determination of the membershipfunction and thus is not used in the subsequent experiments.

Step 3

The TRD is divided using the regular clustering method. As the result ofthe clustering, each learning data is dividable into two segments asshown in Table 2.

                  TABLE 2                                                         ______________________________________                                        INFERENCE RULES                                                                              LEARNING DATA NUMBERS                                          ______________________________________                                        R.sup.1        1, 2, 3, 4, 5, 11, 12, 13, 14, 15                              R.sup.2        6, 7, 8, 9, 10, 16, 17, 18, 19, 20                             ______________________________________                                    

Step 4

In order to infer with the value of W^(s) ε[0,1] a proportion W_(i) ^(s)={0, 1} belonging to learning data (x_(i), y_(i)) where i=1, 2, . . . ,20, the three-layered [3×3×3×2] neural network model was learned 5,000times to obtain the fuzzy number A^(s) of the IF part. FIG. 11illustrates the fuzzy number A¹ of the IF part for the inference rule R¹at this time. The input data in Table 2 where μ_(R) ^(') (x_(i),y_(i))=W_(i) ¹ >0 was used as the learning data.

Similarly, the fuzzy number of the IF part for R² was calculated.

Step 5

An inference quantity presuming equation of the THEN part in eachinference rule was calculated. The three-layered [3×8×8×1] neuralnetwork model was learned 20,000 times and θ₄ was obtained as shown inTable 3.

                  TABLE 3                                                         ______________________________________                                        REFERENCE RULE 1: O.sub.4                                                                          27.86                                                    INFERENCE RULE 2: O.sub.4                                                                           1.93                                                    ______________________________________                                    

Steps 6, 7

θ^(s) is calculated where any one input variable is removed from thestructure model of the THEN part of the influence rule R^(s). Athree-layered [2×8×8×1] neural network model was learned 10,000-20,000times. As a result the error sums of squares of FIG. 12 were obtainedfor the inference rules R¹ and R², respectively. The comparison betweensteps 5 and 6 for each inference rule shows that

    all θ.sub.3.sup.1 >θ.sub.4.sup.2 (=27.8)

    θ.sub.3.sup.2 <θ.sub.4.sup.2 (=1.93) where x.sub.1 is removed.

Therefore, the neural network model at step 5 is used as the THEN partmodel for inference rule 1. For inference rule 2, calculation is furthercontinued and is ended at the repetitive calculations at the secondstage. (The neural network model with (x₂, x₃) inputs is used as theTHEN part. The resulting fuzzy model is given by ##EQU8## FIG. 13illustrates the results of the calculation of y^(*) in equation (3)using an inference device which executed the rules thus obtained.

While in the particular embodiment the weighted center of gravitydefined by equation (3) was employed as the quantity of inferenceobtained as the results of the fuzzy inference, a process forcalculating the central value or maximum value may be used instead.While in the particular embodiment the algebraic product was used in thecalculation equation of w^(s), a fuzzy logical operation such as aminiature calculation may be used instead of the algebraic production.

In the steps of the process for determining the inference rule, the stepof determining a non-linear membership function automatically will nowbe described in more detail.

It is natural that the same reference rule will be applied to similarinput data. At step 3, learning data is clustered (FIG. 8(a)). Theseclasses correspond to inference rules in one-to-one relationship. Theinference rule is, for example, "IF x₁ is small and x₂ is large, THEN .. .". Three such inference rules R₁ -R₃ are shown in FIG. 8(a). Onehundred percent of each of the inference rules is obeyed at typicalpoints (a set of input data) in these classes, but the data are moreinfluenced by a plurality of such inference rules nearer the boundary. Afunction indicative of this percentage is the membership function, theform of which is as shown in FIG. 8(c), for example, which is a view ofthe membership function as viewed from above and the hatched portion isan area where the membership functions intersect.

A step 3. 4 is for preparing such membership functions. A neural networkmodel of FIG. 8(b) is prepared, variable values (x1 and x2 in FIG. 8(b)of the IF part are allocated to the input while 1 is allocated to a rulenumber to which the input value belongs and 0 is allocated to rulenumbers other than the former rule number in the output. Learning isthen performed. One of the important features of the neural networkmodel is that a similar input corresponds to a similar output. While inlearning the points x of FIG. 8(a) alone are used, the same output isobtained at points near the points x, namely, the output shows that thesame inference rule is obeyed. The neural network model which hasfinished learning outputs optimal balance percentages belonging to therespective inference rules when it receives data in the vicinity of theboundaries of the respective inference rules. These are the membershipvalues. In other words, the neural network model includes all themembership functions of the respective inference rules.

While in the particular embodiment of the inventive process themembership functions of fuzzy inference rules are determined using thelearning algorithm of the neural network, other learning algorithms maybe used, of course. As will be easily imagined from the fact that theBack-Propagation algorithm described in the algorithms of the neuralnetwork is based on the classic steepest descent method, many learningalgorithms will be easily conceived which are used in the field ofnon-linear minimization problems such as Neuton's method and conjugategradient method. It is commonsense that in the field of patternrecognition a learning method is used in which a standard pattern whichis used for recognition is caused to sequentially approach the inputpattern. Such sequential learning process may be used.

While in the step 5 of the embodiment the THEN part has been illustratedas being constructed by a neural network, it may be expressed usinganother process. For example, if fuzzy modeling such as that describedin a conventional method is used, the THEN part will differ from that ofthe first embodiment in terms of non-linear expression. However, theessence of the present invention will not be impaired whichautomatically determines the fuzzy number of an inference rulesequentially using the learning function if a learning function such asa neural network model is incorporated in the IF part.

In the particular embodiment of the present invention, the expression"automatic formation of a membership function of a fuzzy inference rule"is used. This special case corresponds to a production rule used in aclassic expert system. In such a case, the expression "automaticformation of the reliability of a inference rule" would be moreintelligible. However, this is only a special case of the embodiment ofthe present invention.

A second embodiment of the invention which performs inference on thebasis of the inference rule obtained by the inventive method will now bedescribed. FIG. 9 is a schematic of the first embodiment of theinference device according to the present invention. In FIG. 1,reference numeral 1 denotes a membership value determiner whichcalculates the membership value of the IF part of each inference rule;2, an individual inference quantity determiner which infers a quantityof inference to be calculated at the THEN part of each inference rule;and 3, a final inference quantity determiner which determines the finalquantity of inference from the outputs of the individual inferencequantity determiner 2 and the membership value determiner 1. Referencenumerals 21-2r denote the internal structure of the individual inferencequantity determiner 21 which determines an inference quantity inaccordance with the respective inference rules 1-r. The determiners 1and 21-2r each have a multi-stage network structure such as shown inFIG. 3.

If input values x₁ -x_(m) are input to the inference device of thepresent invention, the membership value determiner 1 outputs therespective percentages in which the corresponding input values satisfythe respective inference rules. The determiner 1 has beforehand finishedits learning in accordance with the steps described in the inferencerule determining process of the present invention. The individualinference quantity determiner 21 outputs an inference quantity whenthose input values obey the inference rule 1. Similarly, the individualinference quantity determiners 21-2r output inference quantities whenthey obey inference rules 2-r. The final inference quantity determiner 3outputs a final inference result from the outputs of the determiners 1and 2. The process for the determination uses the equation (3), forexample.

As will be clear from FIGS. 6 and 7, the input/output relationship ofthe multi-input-single output signal processor 100 which constitutes theneural network model is non-linear. Therefore, the neural network modelitself can perform non-linear inference. Thus, the inference deviceaccording to the present invention realizes a high inference accuracywhich cannot be achieved by linear approximation.

High inference accuracy of the present invention will be illustratedusing a specific application. The application refers to the inference ofthe COD (Chemical Oxygen Demand) density from five parameters measuredin Osaka Bay from April, 1976-March, 1979. The five measured parametersare (1) water temperature (°C.), (2) transparency (m), (3) DO density(ppm), (4) salt density (%), and (5) filtered COD density (ppm). In thisdata, 32 sets of data obtained from Apr., 1976 to Dec., 1978 were usedfor learning and the remaining 12 sets of data were used for evaluationafter the learning process. FIG. 14 illustrates the result (solid line)estimated by the present invention and the actual observed values of CODdensity (broken lines). It will be understood that the presumed resultis a very good one.

As described above, the first embodiment of the inventive inferencedevice includes a membership value determiner constituted by a pluralityof multi-input-single output signal processors connected in at least anetwork such that it has a learning function and is capable ofdetermining a quantity of inference corresponding to the input variablesin a non-linear relationship.

A second embodiment of the inventive inference device is capable ofperforming approximate inference by storing in memory the beforehandobtained input/output relationship of a predetermined neural networkmodel, and referring to the memory instead of execution of neuralnetwork model directly each time this execution is required. While inthe first embodiment the membership value and individual inferencequantity determiners are described as having a neural network modelstructure, an input-output correspondence table may be preparedbeforehand from a neural network model determined by a learningalgorithm if the input variables are obvious beforehand and/or if closeapproximation is permissible. Although the inventive method cannot beincorporated so as to have a learning function in the second embodimentwhich functions as a substitution for the membership value determiner orthe individual inference quantity determiner by referring to thecorrespondence table during execution of the inference rule, aninexpensive small-sized inference device is realized because no circuitstructure such as that shown in FIG. 5 is needed. The inferencecorresponding to non-linearity can be performed as in the firstembodiment of the inference device of the present invention.

As described above, according to the second embodiment, at least one ofthe membership value determiner and the individual inference quantitydeterminer is composed of the memory which beforehand stores theinput/output characteristic of the inference unit constituted by aplurality of multi-input-single output processors connected to at leastthe network, and the memory referring unit. Thus, the inference devicehas a simple circuit structure for determining an inference quantitycorresponding to input variables in a non-linear manner.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, an inferencerule is prepared which has a learning function, forms an inference ruleautomatically and performs the best inference. According to theinventive inference device, inference is performed with high accuracyeven if an object to be inferred is non-linear, so that the device isvaluable from a practical standpoint.

We claim:
 1. A method of determining fuzzy inference rules for anadjusting device to adjust output characteristics of a membership valuedeterminer in an inference device to be used in an outer system,saidinference device comprising a system variable input device for receivinginput signals from said outer system and for generating an input vectorfrom said input signals to the inference device; the membership valuedeterminer for generating a membership value indicative of the degree ofattribution of the input vector to an IF part of a fuzzy inference ruleof an IF-THEN type, the input vector being received from the systemvariable input device; an individual inference quantity determiner forgenerating a first inference quantity of the input vector correspondingto the THEN part of a fuzzy inference rule, the input vector beingreceived from the system variable input device; a final inferencequantity determiner for determining a final inference quantity byprocessing values attained from the inference quantities generated bythe individual inference quantity determiner and the correspondingmembership values; and an adjusting device, receiving said input vectorand said observed output from said system variable input device and saidfinal inference quantity, for adjusting output characteristics of themembership value determiner, wherein said membership value determinerincludes a signal processing network having input terminals receivinginput vectors from the system variable input device, output terminalsand an artificial neural network structure including a plurality ofmultiple input-single output signal processors each receiving pluralrespective outputs from one or more others of said signal processors andproviding one output to one or more others of said signal processors;each of the multiple input-single output signal processors having anartificial neural network structure comprising a signal processor whichincludes a memory for storing a plurality of weighting coefficients todefine membership values; a plurality of multipliers for weighting theinput vector from the system variable input device with the weightingcoefficients read from the memory; at least one adder for generatingoutput data by adding plural weighted input vectors from saidmultipliers; and a threshold processor for generating an output as amembership value by clipping the output data from said adders within apredetermined range; the method comprising the steps of:(a) receivingand clustering the input vectors from the system variable input deviceand providing class numbers based on degree of similarity of the inputvector, said class numbers corresponding to the output terminals of themembership value determiner respectively; (b) calculating an idealoutput vector on the assumption that the ideal output vector is outputfrom only the corresponding output terminals of the signal processingnetwork corresponding to the class numbers provided as the result ofclustering; (c) obtaining a difference between the ideal output vectorand said first inference quantity from the signal processing networkwhen said network receives the input vector from the system variableinput device; (d) altering the weighting coefficients stored in thememory in order to decrease the obtained difference; (e) reiterating theabove steps until the difference becomes less than a predetermined valueso that a proper output characteristic is obtained for the signalprocessing network; and (f) fixing the output characteristic of themembership value determiner based on the output characteristics of thesignal processing network after the reiterating step (e) is performed.